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Abstract 

This  research  analyzes  twelve  independent  maintenance  variables  and  one  dependent 
operations  variable  for  three  USAF  bases.  Minitab,  version  15,  and  Excel  are  used  to  analyze 
twelve  months  of  data,  from  Dec  08-Nov  09.  Forward  selection  stepwise  regression  and  the 
best-subsets  procedure  are  used  to  build  predictive  models  of  maintenance  metrics'  effect  on 
homestation  departure  reliability  of  C-17  aircraft  at  Dover,  McGuire,  and  Travis  AFBs.  The 
twelve  independent  maintenance  variables  are  regressed  against  one  output  measure,  departure 
reliability.  The  regression  models  and  validation  results  indicate  regression  models  selection  of 
maintenance  constraints  is  consistent  between  departure  reliability  and  three  independent 
variables:  Total  Not  Mission  Capable  Supply  Rate,  Hourly  Utilization  Rate,  and  Average 
Number  of  Aircraft  Possessed.  The  validity  of  these  findings  is  limited  to  the  time  period 
covered,  but  may  be  generalized  across  C-17  aircraft  at  single-squadron  C-17  bases  within  AMC. 
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I.  Introduction 


Background 

Maintenance  management  metrics — sometimes  called  quality  performance  measures  or 
indicators — are  a  crucial  form  of  information  used  by  maintenance  leaders  to  improve  the 
performance  of  maintenance  organizations,  equipment,  and  people  when  compared  with 
established  goals  and  standards  (HQ  AMC/A4MMP,  2008).  Metrics  provide  a  measurement  of 
performance  and  capability.  Leaders,  supervisors  and  technicians  must  have  accurate  and 
reliable  information  to  make  decisions.  Primary  concerns  of  maintenance  managers  are  how  well 
the  unit  is  meeting  mission  requirements,  how  to  improve  equipment  performance,  identifying 
emerging  support  problems,  and  projecting  future  trends  (HQ  AMC/A4MMP,  2008). 

The  overarching  objective  of  Air  Mobility  Command  (AMC)  maintenance  is  to  maintain 
aircraft  and  equipment  in  a  safe,  serviceable  and  ready  condition  to  meet  mission  needs. 
Maintenance  management  metrics  serve  this  overarching  objective  to  evaluate/improve 
equipment  condition,  personnel  skills,  and  long-term  fleet  health.  Metrics  are  used  at  all  levels  of 
command  to  drive  improved  performance  and  adhere  to  well-established  guidelines  (HQ 
AMC/A4MMP,  2008).  The  AMC  supplement  to  Air  Force  Instruction  (AFI)  21-101,  Aircraft 
and  Equipment  Maintenance  Management  (2008),  states  that  metrics  must  be: 

•  Accurate  and  useful  for  decision-making. 

•  Consistent  and  clearly  linked  to  goals/standards. 

•  Clearly  understood  and  communicated. 

•  Based  on  a  measurable,  well-defined  process. 

Headquarters  AMC  (HQ  AMC)  staff,  primarily  the  Operations  Directorate,  uses  the 
Departure  Reliability  (DR)  formula  to  calculate,  analyze,  and  brief  AMC  leadership  on  a  single 
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unit,  multiple  units,  and/or  the  command’s  mission  reliability  (HQ  AMC/A30C,  2004).  This 
formula  is  the  command  standard.  Departure  Reliability  provides  HQ  AMC  staff  personnel  with 
macro-level  trend  analysis  information  and  helps  identify  potential  failure  points  in  the  mission 
generation  process. 

Problem  Statement 

What  effect  do  maintenance  metrics  in  the  mission  generation  process  have  on  departure 
reliability  (“on-time”  departure  rates)  at  C-17  bases? 

Motivation  &  Implication 

The  researcher  uses  a  background  as  a  former  maintenance  officer  and  mobility  pilot  to 
investigate  an  untapped  association  between  operations  and  maintenance  at  three  C-17  bases. 

The  goal  of  this  paper  is  to  analyze  performance  measures  spanning  the  entire  mission  generation 
process,  from  preparation  to  launch,  so  base-level  commanders  can  improve  departure  reliability. 

API  10-202  Volume  6  contains  conflicting  information.  According  to  the  regulation, 
"local  commanders  will  not  use  the  DR  formula  to  assess  their  ability/inability  to  produce  on- 
time  mission  departures"  (HQ  AMC/A30C,  2004,  p.  9);  however,  they  are  to  use  mission 
reliability,  or  DR  data,  to  "assess  internal  processes  which  affect  their  station’s  ability/inability  to 
produce  on-time  mission  departures"  (HQ  AMC/A30C,  2004,  p.  8).  Both  API  10-202  and  21- 
101  direct  base-level  commanders  to  monitor  and  analyze  their  respective  performance 
measures,  but  neither  regulation  specifically  delineates  how  to  intermix  efforts  between 
maintenance  and  operations  to  increase  DR,  the  supposed  focus  of  the  entire  mission  generation 
process. 

According  to  the  APLMA  Maintenance  Metrics  Handbook  (2009),  the  flying 
schedule  is  the  foundation  for  metrics  and  it  sets  the  pace  for  the  entire  wing: 

It  (flying  schedule)  must  be  built  on  sound  principles  that  are  clearly  articulated 
and  vigorously  defended  by  wing  leadership.  The  flying  schedule  is  an  important 
document  that  drives  consumption  of  Air  Porce  resources,  and  the  sortie  is  the 
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focal  point  of  consumption.  Maintainors  must  focus  on  the  sortie  and  all  the 
events  required  for  it  to  sueeeed.  A  schedule  is  established  to  attempt  a  smooth 
flow  of  resource  use  that  ineludes  people,  aircraft,  and  consumables.  Without  a 
flying  sehedule,  all  moving  parts  eertainly  would  not  come  together  efficiently  (p. 
8). 

Research  Focus 

This  research  focuses  on  bases  within  AMC  that  host  a  single  C-17  squadron.  Bases 
include;  MeGuire  AFB,  NJ;  Travis  AFB,  CA;  Dover  AFB,  DE.  Each  base  owns  multiple  major 
weapons  systems  (MWS)  squadrons  and  eaeh  base  faees  similar  challenges  in  maintenance  and 
launch  capabilities.  Expanding  the  researeh  to  wings  with  multiple  C-17  squadrons,  such  as 
McChord  AEB  and  Charleston  AFB,  does  not  capture  the  difficulties  faced  by  units  with 
dissimilar  aircraft.  For  example,  maintenance  and  logistics  resources  are  unique  limitations  at 
Travis  AFB  where  C-5s,  C-17s,  and  KC-lOs  are  parked  on  the  flightline.  Charleston  and 
McChord  AFBs  have  the  ability  to  consolidate  efforts  and  coneentrate  their  force  among  a  single 
MWS,  the  C-17. 

Assumptions/Limitations 

In  this  researeh  paper,  only  C-17  data  is  compiled  from  three  mobility  wings.  DR 
measurability  is  focused  on  unit  controllable,  homestation  departures.  Enroute  and  transient  DR 
is  not  reflected.  Although  maintenance  metries  are  measured  as  unit  eontrollable,  they  include 
performance  based  off  of  homestation,  enroute,  and  transient  C-17  aircraft. 

Eor  example,  DR  is  calculated  at  McGuire  AEB  for  305  Air  Mobility  Wing  (AMW) 
homestation  C-17  aircraft  departures,  but  does  not  include  KC-10  or  KC-135  data.  However, 
maintenance  metrics  gathered  from  MeGuire  AEB  ineludes  data  from  homestation,  enroute,  and 
transient  C-17  aircraft.  This  includes  C-17s  from  other  Major  Commands  (MAJCOMs)  since 
homestation  maintenanee  support  is  required  of  all  enroute  and  transient  C-17  traffic. 
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DR  measures  total  “on-time”  departure  rates  by  location  regardless  of  cause.  On-time 
refers  to  the  standard  for  departures  contained  within  API  1 1-2C-17  Volume  3,  C-17  Flying 
Operations— those  missions  departing  20  minutes  before  the  scheduled  departure  time  until  14 
minutes  after  the  scheduled  departure  time.  All  qualifying  Tanker  Airlift  Control  Center 
(TACC)  tasked  missions  and  local  training  missions  are  considered. 

Systems  users  who  input  and  analyze  DR  and  maintenance  metrics  are  assumed  to  be 
fully  trained  on  all  systems.  Additionally,  all  data  is  assumed  to  be  reported  in  a  timely  manner 
with  accurate  mission  information  (HQ  AMC/A30C,  2004). 

Research  Objective/Research  Questions 

The  objective  of  this  research  is  to  use  regression  analysis  to  identify  which  maintenance 
metrics  in  the  mission  generation  process  have  a  relationship  to  departure  reliability.  The  goal  of 
this  paper  is  to  analyze  performance  measures  spanning  the  entire  mission  generation  process, 
from  preparation  to  launch,  so  base-level  commanders  can  improve  departure  reliability.  This 
paper  attempts  to  answer  the  following: 

1)  What  are  the  statistical  relationships  between  maintenance  metrics  and 
departure  reliability? 

2)  What  maintenance  metrics  can  be  focused  on  to  enhance  a  mobility  wing's 
mission  generation  process? 
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II,  Literature  Review 


This  chapter  begins  with  an  overview  of  performance  measures  and  metrics  within  the 
Air  Force.  Speeific  metrics  and  formulas  are  defined.  Next,  a  brief  review  of  regression  analysis 
is  diseussed.  Also,  a  forecasting  model  and  a  previous  Air  Force  Institute  of  Technology  (AFIT) 
thesis  are  studied.  And  finally,  FedEx's  Air  Operations  Division  is  examined  to  see  what  they 
are  doing  to  tie  their  maintenanee  and  operations  together. 

Air  Force  Metrics 

Operating  eonditions  create  the  organizational  environment  within  whieh  management 
establishes  strategies  and  taeties  to  achieve  goals  (Coyle,  Bardi,  &  Novack,  2006).  As  the  Air 
Force  defines  its  future,  it  is  eritieal  that  they  continue  to  improve  its  effieieney  and  effeetiveness 
(AFLMA,  2009).  An  essential  element  for  this  evaluation  is  metries.  According  to  the  Air  Foree 
Logistics  Management  Ageney  (2009),  “Metries  is  not  a  bad  word,  but  a  tool  for  gauging  where 
your  foeus,  as  a  maintainer,  needs  to  be  direeted.  Good  metrics  are  measurable  and  ean  be 
mapped  to  goals,  both  strategic  and  tactical”  (p.  3).  When  it  eomes  to  aireraft  maintenanee,  the 
Air  Foree  depends  on  metries  to  judge  whether  or  not  they  are  measuring  up  to  the  standard  and 
sueeeeding  in  its  maintenanee  efforts  (Pendley,  2008). 

Agile  organizations  are  essentially  information  proeessing  entities  (Desouza  &  Hensgen, 
2005).  The  performanee  of  an  organization  will  depend  on  how  effectively  and  effieiently  it 
processes  information  from  its  internal  and  external  environments  and  applies  the  information 
towards  realization  of  its  goals  and  objeetives.  As  noted  by  Desouza  and  Hensgen  (2005),  an 
agile  organization  will  be  able  to  (1)  sense  signals  (data)  in  the  environment,  (2)  proeess 
(eonstruct  information)  them  adequately,  (3)  mobilize  knowledge-based  resources  and  processes 
to  take  advantage  of  future  opportunities,  and  (4)  continuously  learn  and  improve  the  operations 
of  the  organization.  “In  addition,  an  agile  organization  will  undertake  these  activities  in  quiek 
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time  cycles  and  with  minimal  cost  and  effort”  (p.  26).  Optimal  information  processing  is  a 
critical  element  for  building  agile  maintenance  organizations  (Desouza  &  Hensgen,  2005). 

Sources  defining  metrics  and  which  speak  to  its  significance  within  the  USAF 
maintenance  community  are  vast.  MAJCOM  instructions,  agency  handbooks,  policy  directives, 
and  AFIT-sponsored  research  are  extensive  albeit  complicated,  confusing,  and  sometimes 
contradictory.  The  majority  of  metrics  are  tied  solely  to  individual  AMC  directorates,  without 
tying  them  across  directorates. 

As  an  example,  the  AMC  Fixed  Command  and  Control  Operations  Branch,  AMC/A30C, 
uses  only  two  metrics:  Departure  Reliability  and  Deviation  Accountability  Rate  (DAR). 
However,  the  AMC  Maintenance  and  Logistics  Directorate,  AMC/A4,  uses  a  wide  range  of 
metrics.  A  few  of  those  are  listed  in  Table  1. 

12-Hour  Fix  Rate 
Aircraft  Availability 
Average  Number  of  Aircraft  Possessed 
Break  Rate 
Cannibalization  Rate 
Delayed  Discrepancies  Rate 
Hourly  Utilization  Rate 
Mission  Capable  Rate 

Total  Not  Mission  Capable  for  Maintenance  Rate 
Total  Not  Mission  Capable  for  Supply  Rate 

Table  1,  Primary  maintenance  metrics 
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Fix  Rate  (FR).  The  FR  is  a  leading  indicator  showing  how  well  the  repair  process  is  being 
managed.  It  is  a  percentage  of  aircraft  with  a  landing  status  code  of  3  (includes  system  cap 
codes  3  and  4)  returned  to  a  flyable  status  in  a  certain  amount  of  time  (clock  hours). 

Problems  found  by  maintenance  after  the  aircraft  lands  (ground  found)  are  not  considered  in  the 
fix  time.  The  fix  time  stops  when  all  Landing  Status  Code  3  Pilot  Reported  Discrepancies 
(PRDs)  are  fixed  even  if  the  aircraft  remains  Not  Mission  Capable  (NMC).  This  metric  is  an 
excellent  tool  to  track  "dead  time"  in  aircraft  repair  processes  because  it  measures  the  speed  of 
repair  and  equipment  maintainability.  The  common,  standard  interval  for  this  metric  is  12-hours 
(HQ  AMC/A4MMP,  2008). 

Tt  (*  o)  =  "Code-3  "  Breaks  pLxed  Witfam  12  Koiirs  of  Landmg  x  IOC 
_ Total  "Code-?"  Bleak  3 _ 

Aircraft  Availability.  Percentage  of  a  fleet  not  in  a  Depot  possessed  status  or  NMC  aircraft  (that 
are  unit  possessed).  The  metric  may  be  created  at  the  MWS  level  or  may  be  grouped  by  fleet 
(e.g.,  Aggregate,  Bombers,  Fighters)  to  determine  “Aircraft  Availability”  (HQ  AMC/A4MMP, 
2008). 

J.3.1.  Availability  rate  =  MC  houn*  x  100 

Total  Possessed  hours** 


Average  Number  of  Aircraft  Possessed.  The  average  number  of  aircraft  possessed  by  a  unit  for  a 
given  period  of  time  (HQ  AMC/A4MMP,  2008). 

Break  Rate  (BR).  The  break  rate  is  a  leading,  flying-related  metric.  It  is  the  percentage  of  aircraft 
that  land  in  “Code-3”  or  “Alpha-3"  for  Mobility  AF  (MAF)  status.  This  metric  primarily 
indicates  aircraft  system  reliability.  It  may  also  reflect  the  quality  of  aircraft  maintenance 
performed.  If  Fix  Rates  are  used  as  a  measurement  of  maintainability,  the  Break  Rate  is  the 
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complementary  measurement  of  reliability.  For  true  evaluation  of  equipment/system  reliability, 
measurements  must  be  taken  at  the  system/subsystem  level.  It  is  also  an  exeellent  predietor  of 
parts  demand.  Several  indieators  that  follow  break  rate  are  Mission  Capable,  Total  Not  Mission 
Capable  for  Supply,  Cannibalization  Rate,  and  Repeat/Reeur  (HQ  AMC/A4MMP,  2008). 

RR  (*/^  =  Niimhpr  of  Snrhp^;  that  T  jinH  'Code-B’'  X  100 
_ Total  Sorbes  Flo^vn 

Cannibalization  Rate  (CR).  The  CR  is  a  leading  indicator  that  reflects  the  number  of 
cannibalization  (CANN)  actions  (removal  of  a  servieeable  part  from  an  aircraft  or  engine  to 
replace  an  unserviceable  part  on  another  aircraft  or  engine  or  to  fill  a  Readiness  Spares  Package 
(RSP)).  In  most  oases,  a  oannibalization  aotion  takes  plaoe  when  the  Logistios  Readiness 
Squadron  (LRS)  cannot  deliver  the  part  when  needed  and  mission  requirements  demand  the 
airoraft  be  returned  to  an  MC  status.  The  CR  is  the  number  of  oannibalization  actions  per  total 
sorties  flown.  This  rate  inoludes  all  aircraft-to-airoraft,  engine-to-aircraft,  and  airoraft/engine  to 
RSP  oannibalization  aotions.  Sinoe  LRS  relies  on  the  baok  shops  and  depot  for  replenishment, 
this  indicator  can  also  be  used,  in  part,  to  indioate  baok  shop  and  depot  support  (HQ 
AMC/A4MMP,  2008). 

CR  f*o')  -  Number  of  Aircraft  anH  Fntnnp  CANNs  x  100 
_ Total  Sonigs  Flown 


Deferred  (or  Delayed)  Diserepaney  (DD)  Rate  (DDR).  Non-grounding  disorepancies  should  be 

transferred  from  the  AFTO  Form  781 A  to  the  78 IK.  Preplanned  time  ohanges  and  TCTOs  are 

not  oonsidered  delayed  until  the  soheduled  day  for  oompletion  is  past  and  the  action  is  not 

completed  (HQ  AMC/A4MMP,  2008). 

Total  DDR  (®  i)  =  Total  (Snapshot)  .WM  -  AUT  Discrepancies 
_ A%~eragc  Aircraft  Possessed _ 
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Hourly  Utilization  (UTE)  Rate.  The  UTE  rate  is  a  leading  indicator,  but  serves  as  a  yardstick 
for  how  well  the  maintenance  organization  supports  the  unit’s  mission.  The  UTE  rate  is  the 
average  number  of  sorties  or  hours  flown  per  Primary  Aerospace  vehicle  Authorized  (PAA)  per 
month  as  found  in  the  HAP/A30PB  documents.  This  measurement  is  primarily  used  by 
operations  in  planning  the  unit's  flying  hour  program.  Maintenance  uses  this  measurement  to 
show  usage  of  assigned  aircraft.  Since  UTE  rates  are  used  for  planning,  actual  UTE  rates 
(computed  at  the  end  of  the  month)  are  used  to  evaluate  the  unit's  monthly  accomplishment 
against  the  annual  plan.  Typically,  Combat  AE  (CAE)  units  measure  the  sortie  UTE  rate,  while 
MAE  units  measure  the  hourly  UTE  rate  to  more  accurately  measure  the  combined  performance 
of  operations  and  maintenance  (HQ  AMC/A4MMP,  2008). 

UTE  Rate  =  Somes  ( or  hours )  Flown  per  Month 

Mission  Capable  (MC)  Rate.  Though  this  is  a  lagging  indicator,  the  MC  rate  is  perhaps  the  best- 
known  yardstick  for  measuring  a  unit’s  performance.  It  is  the  percentage  of  possessed  hours  for 
aircraft  that  are  Full  Mission  Capable  (EMC)  or  Partial  Mission  Capable  (PMC)  for  specific 
measurement  periods  (e.g.,  monthly  or  annual).  A  low  MC  rate  may  indicate  a  unit  is 
experiencing  many  hard  breaks,  parts  supportability  shortfalls,  or  workforce  management  issues. 
Maintenance  managers  should  look  for  workers  deferring  repairs  to  other  shifts,  inexperienced 
workers,  lack  of  parts  from  ERS,  poor  in-shop  scheduling,  high  CANN  rates,  or  training 
deficiencies.  High  commitment  rates  may  also  contribute  to  a  lower  MC  rate.  The  key  is  to  focus 
on  negative  trends  and  identify  systemic,  underlying  causes.  Further,  the  root  factors  of  the  MC 
rate  should  be  measured,  evaluated,  and  reported  through  the  use  of  the  TNMCM,  TNMCS,  and 
NMCB  rates  (HQ  AMC/A4MMP,  2008). 
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MC  {•;)  =  FMCHoun  + PMC  Hours  x  100 
Possessed  Hours 

Total  Not  Mission  Capable  Maintenance  (TNMCM)  Rate.  Though  a  lagging  indicator,  the 
TNMCM  rate  is  perhaps  the  most  eommon  and  useful  metrie  for  determining  if  maintenanee  is 
being  performed  quiekly  and  aeeurately.  It  is  the  average  pereentage  of  possessed  aireraft 
(ealeulated  monthly/annually)  that  are  unable  to  meet  primary  assigned  missions  for  maintenance 
reasons.  Any  aireraft  that  is  unable  to  meet  any  of  its  wartime  missions  is  eonsidered  NMC.  The 
TNMCM  is  the  amount  of  time  aircraft  are  in  NMCM  plus  Not  Mission  Capable  Both  (NMCB) 
status.  Maintenanee  managers  should  look  for  a  relationship  between  other  metries  sueh  as  R/R, 
BR  and  FR  to  the  TNMCM  Rate.  A  strong  eorrelation  eould  indicate  heavy  workloads  (e.g., 
people  are  over  tasked),  poor  management,  training  problems  or  poor  maintenanee  praetiees.  The 
TNMCM  is  also  ealled  “out  for  maintenance.” 


TN^.ICM  Co'S  =  NMCM  Hrs  NMC:B  Hts  x  100 

Possessed  Hours 

Total  Not  Mission  Capable  Supply  (TNMCS)  Rate.  Though  this  lagging  metric  may  seem  a 
“LRS  responsibility”  beeause  it  is  prineipally  driven  by  availability  of  spare  parts,  it  is  often 
directly  indicative  of  maintenance  praetiees.  For  instanee,  maintenance  ean  keep  the  rate  lower 
by  eonsolidating  feasible  CANN  aetions  to  as  few  aireraft  as  praetieal.  This  monthly/annual 
metrie  is  the  average  percentage  of  possessed  aircraft  that  are  unable  to  meet  primary  missions 
for  supply  reasons.  The  TNMCS  rate  is  the  time  aireraft  are  in  NMCS  plus  NMCB  status. 
TNMCS  is  based  on  the  number  of  airframes  out  for  mission  capable  (MICAP)  parts  that  prevent 
the  airframes  from  performing  their  mission  (NMCS  is  not  the  number  of  parts  that  are  MICAP). 
Maintenance  managers  must  closely  monitor  the  relationship  between  the  CR  and  TNMCS. 
TNMCS  is  also  ealled  "out  for  supply." 
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rNMCS  r = m^ks  Hh  -  nmcb  Hts  x  loo 

Possessed  Hours 

These  previous  performance  measures  are  a  sample  of  AMC/A4  metrics,  however  they 
differ  from  what  AMC/A30C  uses  and  from  what  the  Mission  Reliability  and  Reporting  System 
(MRSS)  employs.  AMC/A30C  uses  MRSS  to  provide  United  States  Transportation  Command, 
AMC,  other  MAJCOMs,  and  unit  commanders  with  the  information  necessary  to  conduct 
command  and  control  (C2)  and  the  ability  to  assess  and  improve  the  health  of  the  air  mobility 
component  of  the  Defense  Transportation  System  (HQ  AMC/A30C,  2004). 

Departure  Reliability  reporting  is  another  example  illustrating  the  divergence  in  reporting 
procedures  between  maintainers  and  operators.  For  maintainers.  Logistics  DR  is  the  percent  of 
departures  that  are  delayed  because  of  supply,  saturation,  or  maintenance  problems,  providing  a 
measurement  up  to  the  point  of  an  aircraft’s  launch  regarding  logistics  parameters  (AFLMA, 
2009).  Conversely,  for  operators,  DR  is  a  metric  from  preparation  to  execution,  reflecting  only  if 
the  aircraft  launches  on  time,  without  consideration  for  deviations  within  the  formula  (HQ 
AMC/A30C,  2004). 

Logistics  DR  =  ((#  of  Departures  -  #  of  Logistics  Delays)  /  (#  of  Departures))  x  100 
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LAUNCH  SEQUENCE  OF  EVENTS 


PREPARATION 


Measurement:  Launch 
(Departure  Reliability) 


t 


EXECUTION 


MISSION  EXECUTION 


Measurement: 
Mission  Completion 

(Arrival  Reliability) 


AIRBORNE  PHASE  (ABP) 


t 


CLOSURE 


Figure  1,  Mission  Launch  and  Execution  Model  (HQ  AMC/A30C,  2004) 

AMC/A30C  uses  DR  and  DAR  as  mission  reliability  formulas  to  calculate,  analyze,  and 

brief  AMC  leadership  on  a  single  unit,  multiple  units,  and/or  the  command’s  mission  reliability 
(HQ  AMC/A30C,  2004).  DR  is  calculated  as  the  number  of  on  time  departures  divided  by  the 
total  departures  for  qualifying  missions  multiplied  by  100.  It  is  similar  to  Logistics  DR,  but  does 
not  specifically  take  delays  into  account  in  its  formula.  Instead,  deviations  and  delays  are 
calculated  in  a  different  metric,  DAR. 


DR  =  (On  Time  Departures/Total  Departures)  *  100 


DAR  =  (Accountable  Deviations  /  Departures)  *  100 


Thus,  a  coherent  metric  does  not  exist  tying  the  maintenance  and  operations  processes  together, 
from  scheduling  to  sortie  launch,  demonstrating  the  gap  that  is  still  present  among  two 
competing  philosophies. 

Regression  Analysis 

Managerial  decisions  are  often  based  on  the  relationship  between  two  or  more  variables. 
Sometimes  a  manager  will  rely  on  intuition  to  judge  how  these  variables  are  related.  However,  if 
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data  can  be  obtained,  a  statistical  procedure  called  regression  analysis  can  be  used  to  develop  an 
equation  showing  how  the  variables  are  related  (Anderson,  Sweeney,  &  Williams,  2008). 

Regression  analysis  cannot  be  interpreted  as  a  procedure  for  establishing  a  cause-and- 
effect  relationship  between  variables.  "It  can  only  indicate  how  or  to  what  extent  variables  are 
associated  with  each  other"  (Anderson,  Sweeney,  &  Williams,  2008,  p.  548).  Any  conclusions 
about  cause-and-effect  must  be  based  upon  the  judgment  of  individuals  most  knowledgeable 
about  the  application.  Therefore,  concluding  a  cause-and  effect  relationship  is  warranted  only  if 
the  analyst  can  provide  some  type  of  theoretical  justification  that  the  relationship  is  in  fact  causal 
(Anderson,  Sweeney,  &  Williams,  2008). 

Previous  AFIT  Thesis 

Very  few  sources  were  found  during  the  research  and  literature  review  process  that 
specilieally  deal  with  mobility  airlift  assets  with  respect  to  metrics  and  regression  analysis. 
However,  an  AFIT  thesis  written  by  Captain  Charles  R.  Jung  in  1991  came  closest  to  mirroring 
this  research's  proposed  topic. 

As  the  problem  statement  in  his  thesis,  Jung  (1991)  writes,  "Existing  production 
capability  measurements  in  aircraft  maintenance  fail  to  give  Strategic  Air  Command  (SAC) 
maintenance  managers  an  accurate  estimation  of  maintenance  production  capability  when 
planning  maintenance  support  for  sortie  generation"  (p.  3). 

Jung  (1991)  concludes  that  "the  most  appropriate  forecasting  technique  for  this  thesis 
application  is  a  regression  model  based  on  the  need  for  identification  of  variable  relationships 
between  maintenanee  constraints  and  production  output"  (p.  25). 

Unfortunately,  Jung  (1991)  finds  his  results  to  be  "inconclusive  as  to  what  maintenance 
constraints  are  indicators  of  production  capability  in  aircraft  maintenance."  Furthermore,  he 
contends  that  "Maintenance  production  is  a  complex  dynamic  system  that  is  not  easily  definable 
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in  terms  of  production  inputs  and  outputs  and  makes  maintenance  performance  measurement 
difficult  at  best"  (p.  116). 

Similar  to  Jung,  the  researcher  examined  a  MAJCOM's  (AMC)  maintenance  metrics. 
Also,  regression  analysis  was  applied  for  possible  relationships  between  independent 
maintenance  variables  and  DR,  the  dependent  variable. 

Jung  (1991)  proposes  the  following  research  questions: 

1)  What  are  the  existing  measures  of  aircraft  maintenance  production  capability  in  SAC? 

2)  What  are  the  aircraft  maintenance  production  constraints  that  limit  or  enhance 

production  capability? 

3)  What  are  the  statistical  relationships  between  the  maintenance  constraints  and  an 

organization's  production  capability? 

4)  What  maintenance  constraints  can  be  used  in  a  predictive  model  of  a  maintenance 

organization's  sortie  producing  capability  (p.  3)? 

However,  unlike  Jung,  this  research  centers  on  the  association  between  maintenance  and 
operations  organizations.  Also,  Jung's  scope  of  research  includes  maintenance  performance 
measures  among  several  different  SAC  aircraft  across  numerous  wings,  whereas  this  research 
involves  a  single  MWS  at  only  three  wings. 

Jung's  suggested  future  research  states:  The  results  of  previous 
research. .  .indicate  that  future  research  in  this  area  at  the  aggregate  level  may  not 
be  appropriate.  Research  at  a  lower  level,  such  as  one  particular  aircraft  serial 
number  of  an  aircraft  system  type  using  a  significantly  larger  data  sample  over  a 
longer  period  of  time  may  prove  to  be  profitable.  The  larger  sample  may  reduce 
the  random  and  cyclical  variance  that  hindered  this  research.  The  methodology 
used  in  this  thesis  appears  sound  and  could  help  in  any  future  efforts,  (pp.  115- 
116) 

This  research  follows  Jung's  suggestions  to  research  at  a  lower  level  with  a  significantly  larger 
data  sample;  however,  the  research  encompasses  a  slightly  shorter  12  month  period  versus  the  15 
month  timeframe  Jung  uses. 
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Air  Force  Forecasting  Model 

Developing  a  model  may  be  an  appropriate  fit  under  certain  situations  and  deliver  some 
degree  of  predictability;  however,  models  used  in  forecasting  are  sometimes  flawed.  According 
to  Oliver,  Johnson,  White,  and  Arostegui  (2001),  the  Air  Force  uses  the  Funding/ Availability 
Multimethod  Allocator  for  Spares  (FAMMAS)  model  to  forecast  overall  MC  rates  for  each 
aircraft  in  its  inventory.  FAMMAS  uses  time-series  forecasting  techniques  to  predict  overall  MC 
rates  for  each  MWS,  using  past,  present,  and  future  spares  funding  levels,  along  with  the  last 
three  years  of  historical  supply  and  maintenance  data.  Numerous  operational  and  funding 
decisions  are  made  each  year  based,  in  part,  on  the  predictions  of  this  model  (p.  3). 

Oliver  et  al.  (2001)  argue  that  while  the  FAMMAS  model  does  an  excellent  job  of 
predicting  MC  rates  for  each  aircraft  based  on  funding  data  and  planning  factors,  it  does  not 
adequately  consider  other  factors  that  could  impact  MC  rates.  Specifically,  the  FAMMAS  model 
does  not  incorporate  any  logistics  or  operations-related  factors  into  its  prediction  computations  of 
MC  rates,  other  than  TNMCM/S  data  that  act  as  adjustment  factors  in  the  model  (Oliver, 

Johnson,  White,  &  Arostegui,  2001). 

Past  studies  have  identified  many  other  factors  related  to  MC  rates.  “Unfortunately,  there 
have  been  few  attempts  to  include  these  different  factors  in  the  construction  of  a  mathematical 
model  that  explains  and  forecasts  MC  rates”  (Oliver  et  ah,  2001,  p.  31).  While  FAMMAS  is  an 
effective  tool  for  predicting  MC  rates,  it  does  not  adequately  consider  other  significant  factors 
besides  funding.  Furthermore,  it  does  not  identify  potential  cause-and-effect  relationships, 
especially  between  operations  and  logistics,  which  might  be  manipulated  to  affect  future  MC 
rates  (Oliver,  Johnson,  White,  &  Arostegui,  2001). 

Outside  Practice-FedEx 
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Both  the  Air  Force  and  FedEx  are  interested  in  consolidating  performance  measures  into 
meaningful  metrics  using  the  balanced  scorecard,  developed  by  Kaplan  and  Norton  in  the  early 
1990s  as  a  tool  to  translate  an  organization’s  vision  and  strategy  into  a  coherent  or  balanced  set 
of  performance  measures  (Johnson,  2007).  “While  traditional  methods  focus  only  on  financial 
measurements  to  gauge  organizational  performance,  balanced  scorecard  measures  focus  on  four 
key  areas:  customer,  business  processes,  financial,  and  learning  and  growth”  (Johnson,  2007,  p. 
2).  The  performance  measures  link  activities  at  all  levels  to  the  organization’s  strategic 
objectives  through  a  continuous  chain  of  cause-and-effect  relationships  (Johnson,  2007).  This 
permits  managers  at  every  level  to  monitor  how  other  levels  are  performing  and  allows 
employees  at  each  level  to  see  how  their  efforts  contribute  to  the  organization’s  overall  strategic 
objectives  (Kaplan  &  Norton,  1996). 

Johnson  (2007)  states,  “Prior  to  implementation  of  the  balanced  scorecard.  Air  Force 
metrics  were  heavily  focused  on  financial  and  production  indicators  and  not  on  providing 
support  to  the  warfighter”  (p.  3).  The  balanced  scorecard  initiative  was  adopted  to  correct  this 
deficiency  and  to  develop  a  broader,  more  holistic  view  of  organizational  effectiveness,  focused 
on  providing  the  best  possible  support  to  the  warfighter  (Johnson,  2007).  Once  the  logistics 
balanced  scorecard  has  been  fully  implemented  within  the  Air  Force,  it  will  provide  the  ability  to 
view  the  health  and  welfare  of  Air  Force  logistics  in  a  single  location  with  drill-down  capability 
to  support  management  analysis  and  decision-making  (Johnson,  2007). 

FedEx  Express  is  an  example  of  a  civilian  company  attempting  to  establish  a  balanced 
scorecard  by  means  of  the  Line  Maintenance  Scorecard.  Larry  Crull,  FedEx  Air  Operations 
Division  manager  and  Lean  Ops  Development  &  Education  Black  Belt,  was  asked  for  his 
personal  opinion  on  the  link  between  maintenance  and  operations  metrics.  During  an  interview 
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at  the  FedEx  Express  headquarters  in  Memphis,  TN,  Larry  Crull  was  asked,  "Is  there  a 

relationship  between  your  division's  metrics  and  the  flightline's  to  make  FedEx  more  efficient?" 

Crull  answered,  “We  have  in  the  past  had  a  weak  relationship  with  our  operations 
side  but  we  are  now  improving  the  relationship  and  working  jointly  to  improve 
our  processes  where  the  operations  touch.  This  has  been  a  shift  for  some  to  look  at 
the  other  operations  as  our  customer/suppliers  and  has  been  a  slow  but  vital 
journey."  (personal  communication,  November  4th,  2009) 

Below  are  slides  provided  to  the  researcher  by  Larry  Crull  from  a  "fictional"  FedEx  Line 

Maintenance  Scorecard  (September  16th,  2009): 


Line  Maintenance  Scorecard 


Air  Operations  Division 
AOD  Performance  Management 
September  16,  2009 


Dispatch  Reliability 


Operations 


ConHnueus 

Improvamanl 


I 


Definition 

Parcenlape  el  trunk  revenue  Ibghi  legs  not  havmg  rnamienance  related 
(fespatch  delays  greater  Vian  15  minutes  or  a  cancelaeon  lor  maaitenance 
reasons  Oei^  codesmduded  are  MO,  M3  M3,  M6,  andMA 

•  MO  -  Atrcran  Avaibbiity 

•  M2  -  Unscheduled  Mx 

•  M3  -  Mx  Induced  Human  Factor  Error 

•  M6  -  PusMuKk  -  Tow  Out 

•  MA  -  Mx  After  Block  Out 


Unit  of  Measure 

%  of  Revenue  Fkghls  Legs 


Drilldowns  Available  Custom  Reports 


•  Day  o(  Week  •  Floods  Statiotts 

•  Domesbc !  International 

•  Fleet  /  Senes '  ACN 

•  FkgMType 

•  Route  Type 

•  Sector !  Retpon  /  Disinct  /  Stanon 

•  MEM  Ramp  Sector  /  Departure  Cate 

Page  S 


Figure  2,  FedEx  briefing  title  page 


Figure  4,  Aircraft  Metric:  Dispatch 
Reliability 
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Figure  3,  Key  Measures/Perspectives 


Figure  5,  People  Metric:  Recordable 
Injury  Rate 
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Figure  6,  Quality  Metric:  Repeat  Mx  Figure  7,  Performance  Measure  Data 

Items 

These  slides  are  a  prototype  of  what  is  expected  in  the  near  future  at  FedEx.  Remaining 
development  work  includes  coding  changes  for  maintenance  reorganization  and  final 
testing/validation,  planned  for  November  30th,  2009.  After  final  development  and  roll-out, 
FedEx  plans  to  train  all  central  and  peripheral  users  and  eventually  establish  measurable  targets 
from  historical  data.  Although  still  in  its  infancy,  FedEx  leadership  places  a  great  deal  of 
importance  on  metrics  through  business  process  improvement  and  information  technology. 
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III.  Methodology 


Multiple  regression  analysis  enables  us  to  understand  how  the  dependent  variable,  DR, 
relates  to  the  twelve  independent  maintenanee  variables.  Regression  analysis  is  eondueted  in 
aceordanee  with  Statistics  for  Business  and  Economics,  lOe  (Anderson,  Sweeney,  &  Williams, 
2008).  Data  is  eolleeted  over  a  12  month  period,  from  Deeember  01,  2008  until  November  30, 
2009,  and  eompiled  using  Minitab,  version  15.  A  eomputer  program  is  the  only  realistic  means 
of  performing  numerous  computations  required  in  multiple  regression  analysis  (Anderson, 
Sweeney,  &  Williams,  2008). 

Statistical  Analysis 

In  regression  analysis,  the  term  independent  variable  refers  to  any  variable  being  used  to 
predict  or  explain  the  value  of  the  dependent  variable,  but  the  term  does  not  mean  that  the 
independent  variables  themselves  are  independent  in  any  statistical  sense  (Anderson,  Sweeney, 
&  Williams,  2008).  Most  independent  variables  are  correlated  to  some  degree  with  one  another, 
thus  some  degree  of  linear  association  between  independent  variables  exists  (Anderson, 
Sweeney,  &  Williams,  2008).  Multicollinearity  refers  to  the  correlation  among  the  independent 
variables.  Anderson  et  al.  (2008)  warn  that  "if  possible,  every  attempt  should  be  made  to  avoid 
including  variables  that  are  highly  correlated. .  .when  multicollinearity  is  severe,  we  can  have 
difficulty  interpreting  the  results  of  tests  on  the  individual  parameters"  (pp.  644-645). 

Although  there  are  potential  multicollinearity  problems,  this  research  develops  an 
estimated  regression  equation  using  ten  independent  variables  (without  the  dummy  variables)  to 
answer,  in  part,  research  question  1.  To  finish  answering  research  question  1  and  to  answer 
question  2,  forward  selection  regression  and  best-subsets  regression,  using  all  twelve 
independent  variables,  are  used  to  identify  which  independent  variables  provide  the  best  model. 
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Anderson  et  al.  (2008)  summarizes  the  forward  seleetion  regression  procedure  as 
follows: 

The  forward  selection  procedure  starts  with  no  independent  variables.  It  adds 
variables  one  at  a  time  using  the  same  procedure  as  stepwise  regression  for 
determining  whether  an  independent  variable  should  be  entered  into  the  model. 
However,  the  forward  selection  procedure  does  not  permit  a  variable  to  be 
removed  from  the  model  once  it  has  been  entered.  The  procedure  stops  if  the  p- 
value  for  each  of  the  independent  variables  not  in  the  model  is  greater  than  Alpha 
to  enter  (p.  722) 

The  forward  selection  procedure  is  iterative  whereas  the  best-subsets  procedure  is  not  a 
one-variable-at-time  procedure— it  evaluates  regression  models  involving  different  subsets  of  the 
independent  variables  (Anderson,  Sweeney,  &  Williams,  2008).  Forward  selection  does  not 
guarantee  that  the  best  model  for  a  given  number  of  variables  will  be  found;  hence,  this  method 
is  viewed  as  heuristics  for  selecting  a  good  regression  model  (Anderson,  Sweeney,  &  Williams, 
2008). 


Anderson  et  al.  (2008)  describes  best-subsets  regression: 

. .  .this  output  identifies  the  two  best  one-variable  estimated  regression  equations, 
the  two  best  two-variable  equations,  the  two  best  three-variable  equations,  and  so 
on.  The  criterion  used  in  determining  which  estimated  regression  equations  are 
best  for  any  number  of  predictors  is  the  value  of  the  coefficient  of  determination, 
(p.  723) 

The  adjusted  coefficient  of  determination  (R  )  of  both  the  ten-independent-variable 
estimated  regression  equation  and  the  best  model  equation  are  then  compared  to  each  other. 

Next,  to  guard  against  the  potential  high  multicollinearity,  each  selected  independent  variable's 
variance  inflation  factor  (VIF)  from  the  best  model  equation  of  the  forward  selection  procedure 
is  evaluated.  VIF  indicates  the  extent  to  which  multicollinearity  is  present  in  a  regression 
analysis.  Multicollinearity  is  problematic  because  it  can  increase  the  regression  coefficients, 
making  them  unstable  and  difficult  to  interpret.  A  VIF  measures  how  much  the  variance  of  the 
estimated  regression  coefficients  are  inflated  as  compared  to  when  the  predictor  variables  are  not 
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linearly  related  (LEAD  Teehnologies,  Ine.,  2009).  And  finally,  the  forward  seleetion  model  is 
compared  to  selected  models  from  the  best-subsets  procedure  to  determine  the  best  overall 
model. 

Excel  Problems  and  Minitab  Specifics 

Performing  the  regression  analysis  computations  without  the  help  of  a  computer  can  be 
time  consuming  or  nearly  impossible.  Techniques  used  for  regression  analysis  with  Minitab 
were  found  in  the  appendices  of  chapters  14-16  of  Statistics  for  Business  and  Economics,  lOe 
(Anderson,  Sweeney,  &  Williams,  2008).  Initially,  this  researcher  attempted  to  use  Excel; 
however.  Excel  is  not  equipped  standard  with  a  stepwise  regression  tool  in  its  data  analysis 
regression  package.  Consequently,  a  special  software  package  add-in  (Pekoz,  2009)  from  the 
internet  was  downloaded  and  installed.  After  several  compilations,  several  numerical 
inaccuracies  were  found  in  Excel  as  demonstrated  by  outputs  as  "#NUM!"  and  "65535",  an 
Excel-ism  for  a  t-statistic  equal  to  infinity.  This  is  an  indication  that  the  numerical  methods  used 
by  Excel  are  out-of-date,  and  cannot  be  trusted  (Simonoff,  2008).  Simonoff  argued  that  "these 
problems  have  been  known  in  the  statistical  community  for  many  years,  going  back  to  the 
earliest  versions  of  Excel. .  .the  solution  to  all  these  problems  is  to  perform  statistical  analyses 
using  the  appropriate  tool— a  good  statistical  package,"  and  he  reasoned  that  many  packages  like 
Minitab  are  available  for  under  $100  with  a  Windows-type  graphical  user  interface  that  are 
remarkably  accurate  and  powerful  (Simonoff,  2008,  p.  6).  Minitab  was  used  almost  exclusively 
after  numerous  compiling  issues  with  Excel's  stepwise  procedure  were  encountered. 

Sources  of  Data 

All  data  is  collected  from  two  U.S.  Government  Information  Systems  (USGIS):  Global 
Data  Support  System  II  (GDSS2)  and  Global  Reach  Eogistics/A4  Information  System.  Both 
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sites  require  strict  logon  protocols  and  permission  rights  before  data  is  obtained.  Monthly  data  is 
gathered  from  three  AMC  bases  over  a  12  month  period.  DR  is  gleaned  from  GDSS2  and 
maintenance  performance  measures  are  collected  using  the  Global  Reach  Logistics/ A4 
Information  System. 

The  primary  MAP  C2  system  for  mission  management  and  movement  reporting  is 
GDSS2  (AMC/A30C,  2004).  GDSS2  is  a  MAP  Porce-Level  C2  automated  system  supporting 
mission  management  and  execution  authority  for  effective  global  air  mobility  mission 
operations.  It  provides  MAP  commanders  accurate,  near  real-time  data  required  for  making 
decisions  concerning  the  deployment,  employment,  and  redeployment  of  MAP  resources. 

GDSS2  interfaces  with  many  other  systems.  Additionally,  GDSS2  provides  all  levels  of 
command  with  the  same  mission  visibility  and  allows  real  time  updates.  It  uses  multiple 
applications  to  access  and  update  the  same  database  while  residing  on  the  “C2  enclave”  which 
contains  other  systems  required  by  the  MAP  such  as  the  Advance  Computer  Plight  Planning 
System  and  the  Integrated  Management  Tool  (AMC/A30C,  2004).  Within  GDSS2  is  the  Reports 
Information  Database  Library  (RIDL),  used  to  display  information  over  a  given  time  period. 


Figure  8,  GDSS2  interface 
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Figure  9,  GDSS2/RIDL  report 

From  the  Global  Reach  Logistics/ A4  Information  System,  the  Situational  Awareness 
Report  Selector  is  drilled  down  into  to  obtain  AMC/A4  maintenance  performance  measures.  The 
information  contained  within  the  Global  Reach  Logistics/ A4  Information  System  is  vast  with  an 
abundance  of  data.  For  example,  reports  are  capable  of  showing  broad,  fleet  wide  health  across 
all  MAJCOMs  or  pared  down  to  display  an  individual  aircraft's  status. 
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Figure  10,  Global  Reach  Logistics/ A4  Information  System 


Situational  Awareness  Report 

Data  Collected  on  11/06/2009 
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Figure  11,  Global  Reach  Logistics/ A4  Information  System/Situational  Awareness  Report 
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IV,  Results  and  Analysis 


Minitab,  version  15,  is  used  for  the  statistieal  analysis  of  the  data.  In  aeeordanee  with  the 
guidelines  developed  by  Anderson  et  al.  (2008),  regression  analysis,  forward  selection,  and  best- 
subsets  regression  are  used  to  answer  the  research  questions.  The  data  is  entered  into  Minitab  to 
test  the  various  regression  models.  Table  2  lists  the  variable  abbreviations  and  their  names. 


Variable  Abbreviation 

Variable  Name 

DR 

Departure  Reliability 

MC 

Mission  Capable  Rate 

TNMCM 

Total  Not  Mission  Capable  Maintenance  Rate 

TNMCS 

Total  Not  Mission  Capable  Supply  Rate 

AvgPoss 

Average  Number  of  Aircraft  Possessed 

AcftAvail 

Aircraft  Availability 

HourlyUse 

Hourly  Utilization  Rate 

CANN 

Cannibalization  Rate 

BreakRate 

Break  Rate 

12HrFix 

12  Hour  Fix  Rate 

DD 

Delayed  Discrepancies  Rate 

Varl 

Dummy  Variable  1 

Var2 

Dummy  Variable  2 

Table  2,  Variable  abbreviations 


Minitab  does  not  allow  a  categorical  factor  to  be  specifically  assigned  a  nominal  value. 
Therefore,  variable  1  and  2  are  created  as  indicator  variables,  also  known  as  dummy  variables. 
Dummy  variables  are  used  to  represent  the  different  levels  of  a  categorical  variable.  It  is  used  to 
model  the  effect  of  qualitative  independent  variables.  A  dummy  variable  may  take  only  the 
value  of  zero  or  one.  When  a  qualitative  variable  has  more  than  two  levels,  as  in  this  research, 
care  must  be  taken  in  both  defining  and  interpreting  the  dummy  variables.  If  a  qualitative 
variable  has  k  levels,  k-1  dummy  variables  are  required,  with  each  dummy  variable  being  coded 
as  0  or  1  (Anderson,  Sweeney,  &  Williams,  2008). 
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This  research  uses  regression  analysis  to  help  predict  DR.  With  DR  as  the  dependent 
variable,  several  independent  variables  are  considered.  The  individual  C-17  bases  are  also  an 
important  factor  in  predicting  the  DR.  Because  each  base  is  a  qualitative  variable  with  three 
levels  (Dover,  McGuire,  Travis),  k-1  or  2  dummy  variables  are  required.  Therefore,  two  new 
factors  are  coded  into  the  data  set  as  follows; 


Base 

Dummy  Variable 

Dover 

00 

McGuire 

1  0 

Travis 

0  1 

Table  3,  Dummy  variables 

Two  dummy  variables  are  required  because  each  base  is  a  qualitative  variable  with  three  levels, 
but  the  assignment  of  0-0,  1-0,  and  0-1  to  indicate  Dover,  McGuire,  and  Travis  is  arbitrary. 

Correlation  Coefficient  Analysis 

In  multiple  regression  analysis,  a  relatively  small  size  of  variables  can  be  advantageous  in 
exploring  introductory  concepts,  but  it  is  difficult  to  illustrate  variable  selection  issues  involved 
in  model  building  (Anderson,  Sweeney,  &  Williams,  2008).  This  research  encompasses  3  bases 
and  considers  12  independent  variables  over  a  12-month  period,  resulting  in  36  data  points  as 
shown  in  Table  4. 
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DR 

MC 
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0 

1 

0.853 

0.894 

0.099 
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0 
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0 
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0.01 

1 
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0 

1 
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0.029 
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11 
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0 
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1 
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0 

1 

0.9 
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0.019 

11 
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0.004 

1 
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0 

1 

0.95 
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0.053 

0.003 

11.8 

0.856 
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0 

0 
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0 

1 

Table  4,  DR  and  12  independent  variables 

As  a  preliminary  step,  a  sample  correlation  is  considered  between  each  pair  of  variables.  Figure 
12  is  the  correlation  matrix  obtained  using  Excel. 
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DR 

MC 

TNMCM 

TNMCS 

AvgPoss 

AcftAvail 

HourlyUse 

CANN 

BreakRate  12HrFix  DD 

DR 

1.000 

MC 

-0.038 

1.000 

TNMCM 

-0.006 

-0.958 

1.000 

TNMCS 

0.176 

-0-675 

0.499 

1.000 
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1.000 
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0.061 

0.447 

-0.433 

-0.358 

0  727 

1.000 
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0.255 
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-0.093 

-0.356 

-0.145 

1.000 

CANN 

0.075 

-0.381 

0.216 

0.516 

0.288 

0.000 

-0.385 

1.000 

BreakRate 

-0.062 

-0.711 

0.731 

0.268 

0.183 

-0.336 

-0.214 

0  229 

1.000 

12HrFix 

0.045 

0.195 

-0.236 

0.057 

-0.115 

0.043 

0.096 

0.075 

-0.034  1.000 

DD 

0.030 

-0.647 

0.571 

0.474 

0  048 

-0.417 

0.147 

0.342 

0.455  -0.078  1.000 

Figure  12,  Correlation  coefficients 

The  correlation  coefficient  between  DR  and  MC  is  -0.038,  between  DR  and  TNMCM  is  -0.006 


and  so  on.  Looking  at  the  correlation  coefficients  between  the  independent  variables,  the 


correlation  between  TNMCM  and  BreakRate  is  0.731;  hence  if  BreakRate  were  used  as  an 


independent  variable,  TNMCM  would  not  add  much  more  explanatory  power  to  the  model.  As  a 
rule  of  thumb,  multicollinearity  can  cause  problems  if  the  absolute  value  of  the  correlation 
coefficient  exceeds  0.7  for  any  two  of  the  independent  variables  (Anderson,  Sweeney,  & 
Williams,  2008). 

Looking  at  the  correlation  coefficients  between  DR  and  each  of  the  independent  variables 
can  give  an  indication  of  which  independent  variables  are,  by  themselves,  good  predictors.  The 
single  best  predictor  of  DR  is  HourlyUse,  because  it  has  the  highest  correlation  coefficient 
(.261).  For  the  case  of  one  independent  variable,  the  square  of  the  correlation  coefficient  is  the 
coefficient  of  determination  (Anderson,  Sweeney,  &  Williams,  2008).  Thus,  HourlyUse  can 
explain  (.261)^(100),  or  6.81%,  of  the  variability  in  DR. 

Estimated  Regression  with  10-Independent  Variables 

Although  there  are  potential  multicollinearity  problems,  an  estimated  regression  equation 
using  all  independent  variables  is  considered.  Results  are  shown  in  Figure  13. 
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The  regression  eguation  is 

DR  =  -  2.70  +  3,74  MC  -  0,05  TNMCM  +  0,64  TNMCS  +  0.296  AvgPoss  -  4.17 
AcftAvail 

+  0.00127  HourlyUse  +0.69  CANN  -  0.10  BreakRate  +  0.0052  12HrFix 
-  0.1S8  DD 


Predictor 

Coef 

SE  Coef 

T 

P 

Constant 

-2,704 

4.216 

-0.64 

0.527 

MC 

3.743 

4.632 

0.81 

0.427 

TNMCM 

-0.053 

1.376 

-0.04 

0.970 

TNMCS 

0.640 

1.205 

0.53 

0.600 

AvgPoss 

0,2960 

0,3014 

0,98 

0.336 

AcftAvail 

-4.172 

4.408 

-0.95 

0.353 

HourlyUse 

0.0012629 

0.0006362 

1.99 

0.057 

CANN 

0.690 

1.185 

0.58 

0.566 

BreakRate 

-0.099 

1.604 

-0.06 

0.951 

12HrFix 

0.00519 

0.02809 

0.18 

0.855 

DD 

-0.1878 

0.2366 

-0.79 

0.435 

S  =  0.0484094 

R-Sq  =  20.6% 

R-Sq(adj)  =  0. 

Analysis  of  Variance 

Source 

DF  SS 

MS  F 

Regression 

10  0.015213 

0.001521  0.65 

Residual  Error 

25  0.058587 

0.002343 

Total 

35  0.073800 

Figure  13,  Minitab  output  for  the  model  involving  all  independent  variables 

The  10-variable  equation  (excluding  Varl  and  Var2)  has  an  adjusted  coefficient  of  determination 


of  0.0%.  However,  the  /j-values  for  the  t  tests  of  individual  parameters  show  that  only 


HourlyUse  is  significant  at  the  a  =  .15  level,  given  the  effect  of  all  the  other  variables. 
Investigating  the  results  obtained  using  just  that  single  variable  is  shown  in  Figure  14. 
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The  regression  eguation  is 
DR  =  0.S20  +  0.000697  HourlyUse 


Predictor  Coef  SE  Coef  T  P 
Constant  0.S2003  0.05439  15.08  0.000 
HourlyUse  0.0006968  0.0004421  1.58  0.124 


S  =  0.0449755  R-Sq  =  6.8%  R-Sq(adj)  =  4.1% 


Analysis  of  Variance 


Source 

DF 

SS 

MS 

F  P 

Regression 

1 

0.005025 

0.005025 

2.48  0.124 

Residual  Error 

34 

0.068775 

0.002023 

Total 

35 

0.073800 

Figure  14.  Minitab  output  for  the  model  involving  HourlyUse 

The  estimated  regression  equation  involving  only  HourlyUse  has  an  adjusted  coefficient  of 


determination  of  4.1%.  It  is  better  than  the  10-independent-variable  estimated  regression,  but 


possibly  still  not  quite  as  good  if  determined  using  other  means. 


So,  how  can  an  estimated  regression  equation  be  found  that  will  do  the  job  given  the  data 


available?  One  approach  is  to  compute  all  possible  regressions.  That  is,  develop  10  one-variable 


estimated  regression  equations,  45  two-variable  estimated  regression  equations  (the  number  of 


combinations  of  10  variables  taken  two  at  a  time),  and  so  on.  In  all,  1,023  different  estimated 


regression  equations  involving  one  or  more  independent  variables  would  have  to  be  fitted  to  the 


data. 


With  Minitab,  or  any  of  a  number  of  excellent  computer  packages  available,  it  is  possible 
to  compute  all  possible  regressions.  Given  a  36  data  point  set  with  12  independent  variables,  two 
variable  selection  procedures,  forward  selection  and  best-subsets  regression,  are  used  to  identify 
which  independent  variables  provide  the  best  model. 

Forward  Selection 
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Figure  15  shows  the  results  obtained  by  using  the  Minitab  forward  proeedure  for  the  data 


using  the  value  of  0.15  fov  Alpha  to  enter. 


Forward  selection.  Alpha-to-Enter:  0.15 


Response  is  DR  on  12  predictors,  with  N  =  36 


Step 

1 

2 

3 

4 

Constant 

0.8963 

0.8661 

0.6688 

0.5045 

7ar2 

0.026 

0.037 

0.047 

0.043 

T -Value 

1.63 

2.24 

2.68 

2.54 

?-Value 

0.113 

0.032 

0.012 

0.016 

INMCS 

0.90 

0.92 

0.92 

I-Value 

1.84 

1.91 

2.00 

P-Value 

0.075 

0.065 

0.055 

AvgPoss 

0.016 

0.022 

T-Value 

1.49 

2.00 

P -Value 

0.146 

0.054 

HourlyDse 

0.00080 

r-Value 

1.83 

P -Value 

0.077 

S 

0.0449 

0.0434 

0.0426 

0.0411 

R-Sq 

7.23 

15.85 

21.32 

28.98 

R-Sq ( adj ) 

4.51 

10.75 

13.95 

19.82 

Hallows  Cp 

-0.4 

-1.4 

-1.2 

-1.8 

Figure  15,  Minitab  forward  selection  output 

In  Figure  15,  s  =  V(MSE)  has  been  reduced  from  .0449  with  the  best  one-variable  model  (Var2) 
to  .0411  after  four  steps.  The  value  of  R-Sq  has  been  increased  from  7.23%  to  28.98%,  and  the 
recommended  estimated  regression  equation  has  an  R-Sq  (adj)  value  of  19.82%,  a  considerable 
increase  compared  to  the  10-independent-variable  regression  equation  using  no  variable  selection 
procedures. 

There  are  four  cases  to  consider  when  using  the  dummy  variables  Varl  and  Var2, 
assuming  that  the  baseline  is  Dover  AFB. 


Scenario 

Result 

1;  Neither  Varl  or  Var2  are  significant 

All  three  bases  are  considered  the  same 

2;  Varl  significant  and  Var2  not  significant 

McGuire  DR  significantly  different  from 

Dover  and  Travis 

3;  Var2  significant  and  Varl  not  significant 

Travis  DR  significantly  different  from  Dover 
and  McGuire 
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4;  Both  Varl  and  Var2  are  significant 


Each  base  is  significantly  different  from  the 
others 


Table  5,  Potential  dummy  variable  scenarios  and  results 


Based  on  the  data,  forward  selection  chose  Var2,  but  not  Varl.  That  means  there  is  no  statistical 


difference  from  the  baseline,  Dover,  and  McGuire,  but  there  is  a  difference  between  Travis  and 


Dover  (Capehart,  2010). 


Best-Subsets 


Figure  16  is  the  Minitab  computer  output  obtained  by  using  the  best-subsets  procedure. 
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Figure  16,  Minitab  best-subsets  regression  output 

The  criterion  used  in  determining  which  estimated  regression  equations  are  best  for  any  number 


of  predictors  is  the  value  of  R-Sq  (Anderson,  Sweeney,  &  Williams,  2008).  For  instance,  Var2, 


with  an  R-Sq  =  7.2%,  provides  the  best  estimated  regression  equation  using  only  one 
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independent  variable;  TNMCS  and  Var2,  with  an  R-Sq  =  15.9%,  provides  the  best  estimated 
regression  equation  using  two  independent  variables.  The  R-Sq  (adj)  =  19.8%  is  largest  for  the 
best  model  with  four  independent  variables;  TNMCS,  AvgPoss,  HourlyUse,  and  Var2.  The 
seeond  highest  R-Sq  (adj),  18.2%,  eontains  five  independent  variables.  Aceording  to  Anderson 
et  al.  (2008),  "All  other  things  being  equal,  a  simpler  model  with  fewer  variables  is  usually 
preferred"  (p.  723). 

Making  the  Final  Choice 

The  analysis  performed  on  the  data  is  good  preparation  for  ehoosing  a  final  model,  but 
more  analysis  should  be  eondueted  before  making  the  final  ehoiee.  As  noted  before,  a  eareful 
analysis  of  the  VIF  should  be  done.  The  following  guidelines  are  used  to  interpret  the  VIF: 


VIF 

Predictors  are,,. 

1<VIF<2 

Not  correlated 

2<VIF<5 

Moderately  correlated 

VIF>5  to  10 

Highly  correlated 

VIF  values  greater  than  10  may  indicate  multicollinearity  is  unduly  influencing  regression  results 


Table  6,  VIF  guidelines 

To  test  the  durability  of  the  forward  seleetion  proeess,  the  VIF  of  TNMCS,  AvgPoss,  HourlyUse, 


and  Var2  are  examined  to  prevent  using  variables  that  are  strongly  negatively  eorrelated  and 


have  a  VIF  greater  than  5.  Figure  17  is  the  eomputer  output  obtained  by  seleeting  variance 


inflation  factors  within  Minitab's  regression  options. 


The  regression  equation  is 

DR=  0.504  +  0.924  TNMCS  +  0.0217  AvgPoss  +  0.000798  HourlyUse  +  0.0433  Var2 


Predictor 

Coef 

SE  Coef 

T 

P 

VIF 

Constant 

0.5045 

0.1571 

3.21 

0.003 

TNMCS 

0.9244 

0.4632 

2.00 

0.055 

1.152 

AvgPoss 

0.02156 

0.01082 

2.00 

0.054 

1.306 

HourlyUse 

0.0007979 

0.0004362 

1.83 

0.077 

1.165 

Var2 

0.04331 

0.01707 

2.54 

0.016 

1.379 

3=0.0411171  R-Sq  =29.0%  R-Sq(adj)  =  19.8% 


Figure  17,  VIF  output 
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Since  TNMCS,  AvgPoss,  HourlyUse,  and  Var2  all  exhibit  a  VIF  elose  to  one,  the  results  of  the 
forward  selection  process  ean  be  used  to  help  ehoose  the  model. 

The  best-subsets  proeedure  shows  that  the  best  four-variable  model  eontains  TNMCS, 
AvgPoss,  HourlyUse,  and  Var2.  This  result  also  happens  to  be  the  four-variable  model 
identified  with  the  forward  seleetion  proeedure.  This  researeher,  in  aeeordanee  with  Maj  Shay 
Capehart  (2010),  Assistant  Professor  of  Statisties  at  AFIT,  contends  that  the  model  appears  to  be 
sound.  The  R-Sq  (adj)  value  is  pretty  low  (19.8%),  but  one  has  to  realize  that  there  is  a  lot  of 
noise  in  this  regression,  meaning  it  is  very  hard  to  prediet  a  large  portion  of  DR. 
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V.  Conclusions  and  Recommendations 


The  results  of  this  researeh  support  evidenee  of  a  moderately  strong  relationship  between 
departure  reliability  and  the  maintenanee  metrics:  Total  Not  Mission  Capable  Supply  Rate, 
Hourly  Utilization  Rate,  and  Average  Number  of  Aircraft  Possessed.  This  means  that  efforts  to 
enhance  the  focus  on  these  metrics  may  have  effectiveness  in  a  larger  effort  to  improve  departure 
reliability.  Consideration  of  why  these  three  performance  measures  are  statistically  relevant 
along  with  the  significance  related  to  other  metrics  within  the  mission  generation  process  must 
be  considered.  Additionally,  this  analysis  has  value  for  current  guidance  as  well  as  potential 
future  research. 

Statistical  Relevance  and  Significance 

TNMCS  is  principally  driven  by  spare  parts  availability.  It  is  based  on  the  number  of 
airframes  out  for  parts.  A  distinct  link  exists  between  TNMCS  and  CANN  rates  (AFLMA, 

2009).  The  best  situation  is  for  both  rates  to  be  as  low  as  possible.  Caution  should  be  exhibited 
not  to  keep  TNMCS  low  at  the  expense  of  increased  CANN  actions— maintenance  should  not  be 
driven  to  make  undesirable  CANNs  just  to  keep  the  TNMCS  rate  low  (AFLMA,  2009). 
According  to  the  AFLMA  (2009),  questions  to  be  asked  when  investigating  the  importance  of 
TNMCS  on  DR  include: 

•  Why  are  so  many  hard-to-get  parts  being  ordered? 

•  Is  the  base-stockage  level  high  enough? 

•  Are  old  parts  being  turned  in? 

•  Could  the  part  be  fixed  on  base? 

•  Even  though  the  current  guidance  says  send  it  back  to  depot,  can  the  status  quo  be 
challenged? 
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Operations  and  maintenance  inherently  both  share  responsibility  of  Hourly  Utilization 
Rate  (UTE)  because  it  reflects  a  combined  performance  (AFLMA,  2009).  If  maintenance  is  not 
meeting  the  UTE  rate,  it  means  the  average  number  of  sorties  per  aircraft  is  lower  than  desired. 
Scheduling  more  sorties  is  not  the  answer  (AEEMA,  2009).  According  to  AEEMA  (2009),  "The 
root  cause  of  a  low  UTE  rate  may  lie  in  maintenance  scheduling  practices  that  result  in  low 
aircraft  availability,  effectiveness  of  the  production  effort  that  repairs  and  prepares  aircraft. .  .or 
even  availability  of  qualified  technicians"  (p.  34).  It  may  also  mean  that  other  factors,  such  as 
weather  or  climate  conditions,  have  an  effect  on  operations  (AEEMA,  2009). 

Averaged  number  of  aircraft  possessed  has  a  significant  corollary  to  other  key 
maintenance  metrics.  A  low  AvgPoss  forces  a  higher  real  UTE  rate  on  fewer  aircraft,  possibly 
compromising  two  key  areas:  scheduled  maintenance  and  deferred  discrepancies.  Higher 
Recur/Repeat  rates  may  occur.  Break  rates  may  also  increase  and  the  Eix  rate  may  suffer  as  well 
(AEEMA,  2009). 

Current  Guidance 

Per  AMCI  I0-202V6  (2004),  base-level  operations  commanders  are  charged  with 
establishing  written  procedures  to  review  and  validate  departure  reliability  information  on  a 
monthly  basis  using  the  following  five-step  process: 


1. 

Detect  a  change  in  reliability  using  the  Deviation  Accountabihty  Rate  formula. 

2, 

Analyze  the  data  to  identifS'  causal  factors  for  the  changes. 

3. 

Document  factors  impacting  reliabihty  and  develop  a  course  of  action  to  improve  departure 
reliability. 

4 

Implement  changes  for  improving  reliability. 

5. 

Return  to  step  rf  1  to  assess  the  effectiveness  of  implemented  changes;  adjust  as  necessarv; 
and  identify  new  factors  affecting  mission  reUability. 

Table  7,  Five-step  departure  reliability  performance  process 
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Additionally,  AFI  21-101  AMCSUPl  (2008)  directs  base-level  maintenance  commanders  to 
manage  the  data  collection  process,  review  data,  and  verify  analysis  for  maintenance  data 
collection  requirements.  Also,  commanders  are  to  evaluate  and  provide  trend  analysis 
information. 

Both  operations  and  maintenance  commanders  are  instructed  with  providing  data  analysis 
for  the  processes  in  which  they  are  responsible;  however,  explicit  guidance  detailing  how  to 
combine  the  analysis  across  the  processes  is  not  given.  As  shown  by  this  paper's  research,  using 
regression  analysis  is  a  step  towards  tying  maintenance  and  operations  together  to  give  AMC  a 
coherent  snapshot  of  a  base's  productivity,  but  also  gives  base-level  commanders  a  means  to 
improve  their  mission  generation  methods. 

Future  Research 

The  research  completed  in  this  paper  does  not  encompass  a  wide  array  of  MWSs,  bases, 
or  MAJCOMs,  but  can  have  significant  relevance  if  used  as  a  springboard  for  similar  situations. 
The  Air  Force  is  consolidating  bases,  creating  fiightlines  full  of  different  aircraft.  For  instance, 
Holloman  AFB,  NM  recently  received  its  first  F-22  Raptor  fighter  aircraft.  Now  Holloman  AFB 
is  home  to  F-22  Raptors,  MQ-1  Predators,  and  MQ-9  Reapers.  Similarly,  Hurlburt  Field,  FL  is 
the  host  base  for  several  aircraft  including  C-130s,  CV-22s,  UH-ls,  and  U-28s.  Holloman  AFB 
is  a  part  of  the  MAJCOM,  ACC  and  Hurlburt  Field  is  owned  by  AFSOC.  These  bases  could 
benefit  from  a  comparable  analysis  in  the  same  manner  this  research  compiled  data  found  at  an 
AMC  MAJCOM  base.  Although  different  maintenance  metrics  may  be  found  to  be  significant, 
the  same  methodology  could  be  applied  to  increase  their  homestation  departure  reliability. 

Although  the  Air  Force  is  creating  more  ramps  with  multiple  MWSs,  "super-bases"  such 
as  McChord  and  Charleston  AFB  will  still  continue  to  exist.  The  effect  of  maintenance  metrics 
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on  DR  at  C-17  super-bases  compared  to  single-squadron  C-17  bases  like  McGuire  may  give  HQ 
AF  a  quantitative  analysis  for  future  basing  decisions.  For  example,  a  favorable  regression 
analysis  at  Charleston  AFB,  shown  over  an  extended  period  of  time,  could  make  a  case  for 
moving  C-17s  from  McGuire  and  Dover  to  Charleston,  consolidating  east  coast  C-17  operations 
at  a  one  base. 

This  research  paper  focuses  on  homestation  DR.  Future  research  of  both  world-wide  and 
enroute  departure  reliability  can  also  be  analyzed.  Introducing  these  two  factors  will  increase  the 
variability  of  the  model  and  must  be  tempered  with  sound  data  mining  to  study  the  maintenance 
parameters  that  directly  affect  that  particular  mission  generation  process.  The  Logistics 
Installations  and  Mission  Support-Enterprise  View  (LIMS-EV)  is  an  application  that  can  be  used 
to  provide  a  single  capability  to  exploit  information  across  all  A4  resources  to  support 
operational,  tactical,  and  strategic  decision  making.  LIMS-EV  supports  all  types  of  reporting 
and  analysis  requirements  using  scorecards  and  dashboards  as  well  as  predictive  analysis 
capabilities  to  all  level  of  users.  LIMS-EV  will  soon  bridge  the  gap  between  legacy  and  current 
data  systems  to  maintain  a  single  enterprise  view  (Curry,  2008). 

Final  Thoughts 

Continuous  improvement  between  maintenance  and  operations  is  not  achievable  without 
sound  performance  measures;  however,  the  data  must  be  analyzed,  interpreted,  and  most 
importantly,  integrated  correctly,  to  make  calculated  decisions.  Pin-pointing  which  maintenance 
metrics  have  the  highest  relationship  to  DR  will  save  time,  labor,  and  money  within  AMC,  thus 
enhancing  the  overall  velocity  and  precision  in  AMC's  airlift  capabilities. 
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